notebook.community

Edit and run 





In [1]:



    


import tensorflow as tf
from tensorflow import data
import shutil
import math
from datetime import datetime
from tensorflow.python.feature_column import feature_column

from tensorflow.contrib.learn import learn_runner
from tensorflow.contrib.learn import make_export_strategy

print(tf.__version__)



    


















    






/Users/khalidsalama/anaconda/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
  return f(*args, **kwds)










    






1.4.0

Steps to use the TF Experiment APIs

  1. Define dataset metadata
  2. Define data input function to read the data from csv files + feature processing
  3. Create TF feature columns based on metadata + extended feature columns
  4. Define an estimator (DNNRegressor) creation function with the required feature columns & parameters
  5. Define a serving function to export the model
  6. Run an Experiment with learn_runner to train, evaluate, and export the model
  7. Evaluate the model using test data
  8. Perform predictions




In [2]:



    


MODEL_NAME = 'reg-model-03'

TRAIN_DATA_FILES_PATTERN = 'data/train-*.csv'
VALID_DATA_FILES_PATTERN = 'data/valid-*.csv'
TEST_DATA_FILES_PATTERN = 'data/test-*.csv'

RESUME_TRAINING = False
PROCESS_FEATURES = True
EXTEND_FEATURE_COLUMNS = True
MULTI_THREADING = True

1. Define Dataset Metadata

  • CSV file header and defaults
  • Numeric and categorical feature names
  • Target feature name
  • Unused columns




In [3]:



    


HEADER = ['key','x','y','alpha','beta','target']
HEADER_DEFAULTS = [[0], [0.0], [0.0], ['NA'], ['NA'], [0.0]]

NUMERIC_FEATURE_NAMES = ['x', 'y']  

CATEGORICAL_FEATURE_NAMES_WITH_VOCABULARY = {'alpha':['ax01', 'ax02'], 'beta':['bx01', 'bx02']}
CATEGORICAL_FEATURE_NAMES = list(CATEGORICAL_FEATURE_NAMES_WITH_VOCABULARY.keys())

FEATURE_NAMES = NUMERIC_FEATURE_NAMES + CATEGORICAL_FEATURE_NAMES

TARGET_NAME = 'target'

UNUSED_FEATURE_NAMES = list(set(HEADER) - set(FEATURE_NAMES) - {TARGET_NAME})

print("Header: {}".format(HEADER))
print("Numeric Features: {}".format(NUMERIC_FEATURE_NAMES))
print("Categorical Features: {}".format(CATEGORICAL_FEATURE_NAMES))
print("Target: {}".format(TARGET_NAME))
print("Unused Features: {}".format(UNUSED_FEATURE_NAMES))



    


















    






Header: ['key', 'x', 'y', 'alpha', 'beta', 'target']
Numeric Features: ['x', 'y']
Categorical Features: ['alpha', 'beta']
Target: target
Unused Features: ['key']

2. Define Data Input Function

  • Input csv files name pattern
  • Use TF Dataset APIs to read and process the data
  • Parse CSV lines to feature tensors
  • Apply feature processing
  • Return (features, target) tensors

a. parsing and preprocessing logic





In [4]:



    


def parse_csv_row(csv_row):
    
    columns = tf.decode_csv(csv_row, record_defaults=HEADER_DEFAULTS)
    features = dict(zip(HEADER, columns))
    
    for column in UNUSED_FEATURE_NAMES:
        features.pop(column)
    
    target = features.pop(TARGET_NAME)

    return features, target

def process_features(features):

    features["x_2"] = tf.square(features['x'])
    features["y_2"] = tf.square(features['y'])
    features["xy"] = tf.multiply(features['x'], features['y']) # features['x'] * features['y']
    features['dist_xy'] =  tf.sqrt(tf.squared_difference(features['x'],features['y']))
    
    return features

b. data pipeline input function





In [5]:



    


def csv_input_fn(files_name_pattern, mode=tf.estimator.ModeKeys.EVAL, 
                 skip_header_lines=0, 
                 num_epochs=None, 
                 batch_size=200):
    
    shuffle = True if mode == tf.estimator.ModeKeys.TRAIN else False
    
    print("")
    print("* data input_fn:")
    print("================")
    print("Input file(s): {}".format(files_name_pattern))
    print("Batch size: {}".format(batch_size))
    print("Epoch Count: {}".format(num_epochs))
    print("Mode: {}".format(mode))
    print("Shuffle: {}".format(shuffle))
    print("================")
    print("")
    
    file_names = tf.matching_files(files_name_pattern)

    dataset = data.TextLineDataset(filenames=file_names)
    dataset = dataset.skip(skip_header_lines)
    
    if shuffle:
        dataset = dataset.shuffle(buffer_size=2 * batch_size + 1)

    #useful for distributed training when training on 1 data file, so it can be shareded
    #dataset = dataset.shard(num_workers, worker_index)
    
    dataset = dataset.batch(batch_size)
    dataset = dataset.map(lambda csv_row: parse_csv_row(csv_row))
    
    if PROCESS_FEATURES:
        dataset = dataset.map(lambda features, target: (process_features(features), target))
    
    #dataset = dataset.batch(batch_size) #??? very long time
    dataset = dataset.repeat(num_epochs)
    iterator = dataset.make_one_shot_iterator()
    
    features, target = iterator.get_next()
    return features, target



    











In [6]:



    


features, target = csv_input_fn(files_name_pattern="")
print("Feature read from CSV: {}".format(list(features.keys())))
print("Target read from CSV: {}".format(target))



    


















    






* data input_fn:
================
Input file(s): 
Batch size: 200
Epoch Count: None
Mode: eval
Shuffle: False
================

Feature read from CSV: ['x', 'y', 'alpha', 'beta', 'x_2', 'y_2', 'xy', 'dist_xy']
Target read from CSV: Tensor("IteratorGetNext:8", shape=(?,), dtype=float32)

3. Define Feature Columns

The input numeric columns are assumed to be normalized (or have the same scale). Otherise, a normlizer_fn, along with the normlisation params (mean, stdv) should be passed to tf.feature_column.numeric_column() constructor.





In [7]:



    


def extend_feature_columns(feature_columns):
    
    # crossing, bucketizing, and embedding can be applied here
    
    feature_columns['alpha_X_beta'] = tf.feature_column.crossed_column(
        [feature_columns['alpha'], feature_columns['beta']], 4)
    
    return feature_columns

def get_feature_columns():
    
    CONSTRUCTED_NUMERIC_FEATURES_NAMES = ['x_2', 'y_2', 'xy', 'dist_xy']
    all_numeric_feature_names = NUMERIC_FEATURE_NAMES.copy() 
    
    if PROCESS_FEATURES:
        all_numeric_feature_names += CONSTRUCTED_NUMERIC_FEATURES_NAMES

    numeric_columns = {feature_name: tf.feature_column.numeric_column(feature_name)
                       for feature_name in all_numeric_feature_names}

    categorical_column_with_vocabulary = \
        {item[0]: tf.feature_column.categorical_column_with_vocabulary_list(item[0], item[1])
         for item in CATEGORICAL_FEATURE_NAMES_WITH_VOCABULARY.items()}
        
    feature_columns = {}

    if numeric_columns is not None:
        feature_columns.update(numeric_columns)

    if categorical_column_with_vocabulary is not None:
        feature_columns.update(categorical_column_with_vocabulary)
        
    if EXTEND_FEATURE_COLUMNS:
        feature_columns = extend_feature_columns(feature_columns)
        
    return feature_columns

feature_columns = get_feature_columns()
print("Feature Columns: {}".format(feature_columns))



    


















    






Feature Columns: {'x': _NumericColumn(key='x', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), 'y': _NumericColumn(key='y', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), 'x_2': _NumericColumn(key='x_2', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), 'y_2': _NumericColumn(key='y_2', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), 'xy': _NumericColumn(key='xy', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), 'dist_xy': _NumericColumn(key='dist_xy', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), 'alpha': _VocabularyListCategoricalColumn(key='alpha', vocabulary_list=('ax01', 'ax02'), dtype=tf.string, default_value=-1, num_oov_buckets=0), 'beta': _VocabularyListCategoricalColumn(key='beta', vocabulary_list=('bx01', 'bx02'), dtype=tf.string, default_value=-1, num_oov_buckets=0), 'alpha_X_beta': _CrossedColumn(keys=(_VocabularyListCategoricalColumn(key='alpha', vocabulary_list=('ax01', 'ax02'), dtype=tf.string, default_value=-1, num_oov_buckets=0), _VocabularyListCategoricalColumn(key='beta', vocabulary_list=('bx01', 'bx02'), dtype=tf.string, default_value=-1, num_oov_buckets=0)), hash_bucket_size=4, hash_key=None)}

4. Define an Estimator Creation Function

  • Get dense (numeric) columns from the feature columns
  • Convert categorical columns to indicator columns
  • Create Instantiate a DNNRegressor estimator given dense + indicator feature columns + params




In [8]:



    


def create_estimator(run_config, hparams):
    
    feature_columns = list(get_feature_columns().values())
    
    dense_columns = list(
        filter(lambda column: isinstance(column, feature_column._NumericColumn),
               feature_columns
        )
    )

    categorical_columns = list(
        filter(lambda column: isinstance(column, feature_column._VocabularyListCategoricalColumn) |
                              isinstance(column, feature_column._BucketizedColumn),
                   feature_columns)
    )

    indicator_columns = list(
            map(lambda column: tf.feature_column.indicator_column(column),
                categorical_columns)
    )
    
    
    estimator = tf.estimator.DNNRegressor(
        
        feature_columns= dense_columns + indicator_columns ,
        hidden_units= hparams.hidden_units,
        
        optimizer= tf.train.AdamOptimizer(),
        activation_fn= tf.nn.elu,
        dropout= hparams.dropout_prob,
        
        config= run_config
    )

    print("")
    print("Estimator Type: {}".format(type(estimator)))
    print("")
    
    return estimator

5. Define Serving Funcion





In [9]:



    


def csv_serving_input_fn():
    
    SERVING_HEADER = ['x','y','alpha','beta']
    SERVING_HEADER_DEFAULTS = [[0.0], [0.0], ['NA'], ['NA']]

    rows_string_tensor = tf.placeholder(dtype=tf.string,
                                         shape=[None],
                                         name='csv_rows')
    
    receiver_tensor = {'csv_rows': rows_string_tensor}

    row_columns = tf.expand_dims(rows_string_tensor, -1)
    columns = tf.decode_csv(row_columns, record_defaults=SERVING_HEADER_DEFAULTS)
    features = dict(zip(SERVING_HEADER, columns))

    return tf.estimator.export.ServingInputReceiver(
        process_features(features), receiver_tensor)

6. Run Experiment

a. Define Experiment Function





In [10]:



    


def generate_experiment_fn(**experiment_args):

    def _experiment_fn(run_config, hparams):

        train_input_fn = lambda: csv_input_fn(
            files_name_pattern=TRAIN_DATA_FILES_PATTERN,
            mode = tf.contrib.learn.ModeKeys.TRAIN,
            num_epochs=hparams.num_epochs,
            batch_size=hparams.batch_size
        )

        eval_input_fn = lambda: csv_input_fn(
            files_name_pattern=VALID_DATA_FILES_PATTERN,
            mode=tf.contrib.learn.ModeKeys.EVAL,
            num_epochs=1,
            batch_size=hparams.batch_size
        )

        estimator = create_estimator(run_config, hparams)

        return tf.contrib.learn.Experiment(
            estimator,
            train_input_fn=train_input_fn,
            eval_input_fn=eval_input_fn,
            eval_steps=None,
            **experiment_args
        )

    return _experiment_fn

b. Set HParam and RunConfig





In [15]:



    


TRAIN_SIZE = 12000
NUM_EPOCHS = 1000
BATCH_SIZE = 500
NUM_EVAL = 10
CHECKPOINT_STEPS = int((TRAIN_SIZE/BATCH_SIZE) * (NUM_EPOCHS/NUM_EVAL))

hparams  = tf.contrib.training.HParams(
    num_epochs = NUM_EPOCHS,
    batch_size = BATCH_SIZE,
    hidden_units=[8, 4], 
    dropout_prob = 0.0)

model_dir = 'trained_models/{}'.format(MODEL_NAME)

run_config = tf.contrib.learn.RunConfig(
    save_checkpoints_steps=CHECKPOINT_STEPS,
    tf_random_seed=19830610,
    model_dir=model_dir
)

print(hparams)
print("Model Directory:", run_config.model_dir)
print("")
print("Dataset Size:", TRAIN_SIZE)
print("Batch Size:", BATCH_SIZE)
print("Steps per Epoch:",TRAIN_SIZE/BATCH_SIZE)
print("Total Steps:", (TRAIN_SIZE/BATCH_SIZE)*NUM_EPOCHS)
print("Required Evaluation Steps:", NUM_EVAL) 
print("That is 1 evaluation step after each",NUM_EPOCHS/NUM_EVAL," epochs")
print("Save Checkpoint After",CHECKPOINT_STEPS,"steps")



    


















    






[('batch_size', 500), ('dropout_prob', 0.0), ('hidden_units', [8, 4]), ('num_epochs', 1000)]
Model Directory: trained_models/reg-model-04

Dataset Size: 12000
Batch Size: 500
Steps per Epoch: 24.0
Total Steps: 24000.0
Required Evaluation Steps: 10
That is 1 evaluation step after each 100.0  epochs
Save Checkpoint After 2400 steps

c. Run Experiment via learn_runner





In [12]:



    


if not RESUME_TRAINING:
    print("Removing previous artifacts...")
    shutil.rmtree(model_dir, ignore_errors=True)
else:
    print("Resuming training...") 


tf.logging.set_verbosity(tf.logging.INFO)

time_start = datetime.utcnow() 
print("Experiment started at {}".format(time_start.strftime("%H:%M:%S")))
print(".......................................") 


learn_runner.run(
    experiment_fn=generate_experiment_fn(

        export_strategies=[make_export_strategy(
            csv_serving_input_fn,
            exports_to_keep=1
        )]
    ),
    run_config=run_config,
    schedule="train_and_evaluate",
    hparams=hparams
)

time_end = datetime.utcnow() 
print(".......................................")
print("Experiment finished at {}".format(time_end.strftime("%H:%M:%S")))
print("")
time_elapsed = time_end - time_start
print("Experiment elapsed time: {} seconds".format(time_elapsed.total_seconds()))



    


















    






Removing previous artifacts...
Experiment started at 14:31:43
.......................................
WARNING:tensorflow:RunConfig.uid (from tensorflow.contrib.learn.python.learn.estimators.run_config) is experimental and may change or be removed at any time, and without warning.
INFO:tensorflow:Using config: {'_task_type': None, '_task_id': 0, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x11e860a58>, '_master': '', '_num_ps_replicas': 0, '_num_worker_replicas': 0, '_environment': 'local', '_is_chief': True, '_evaluation_master': '', '_tf_config': gpu_options {
  per_process_gpu_memory_fraction: 1
}
, '_tf_random_seed': 19830610, '_save_summary_steps': 100, '_save_checkpoints_secs': None, '_log_step_count_steps': 100, '_session_config': None, '_save_checkpoints_steps': 2400, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_model_dir': 'trained_models/reg-model-04'}

Estimator Type: <class 'tensorflow.python.estimator.canned.dnn.DNNRegressor'>

WARNING:tensorflow:RunConfig.uid (from tensorflow.contrib.learn.python.learn.estimators.run_config) is experimental and may change or be removed at any time, and without warning.
WARNING:tensorflow:From /Users/khalidsalama/anaconda/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/monitors.py:267: BaseMonitor.__init__ (from tensorflow.contrib.learn.python.learn.monitors) is deprecated and will be removed after 2016-12-05.
Instructions for updating:
Monitors are deprecated. Please use tf.train.SessionRunHook.

* data input_fn:
================
Input file(s): data/train-*.csv
Batch size: 500
Epoch Count: 1000
Mode: train
Shuffle: True
================

INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Saving checkpoints for 1 into trained_models/reg-model-04/model.ckpt.

* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:31:48
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-1
INFO:tensorflow:Finished evaluation at 2017-11-11-14:31:49
INFO:tensorflow:Saving dict for global step 1: average_loss = 328.465, global_step = 1, loss = 164232.0
INFO:tensorflow:Validation (step 1): average_loss = 328.465, loss = 164232.0, global_step = 1
INFO:tensorflow:loss = 155660.0, step = 1
INFO:tensorflow:global_step/sec: 37.5745
INFO:tensorflow:loss = 185020.0, step = 101 (0.809 sec)
INFO:tensorflow:global_step/sec: 150.261
INFO:tensorflow:loss = 137635.0, step = 201 (0.665 sec)
INFO:tensorflow:global_step/sec: 142.137
INFO:tensorflow:loss = 171515.0, step = 301 (0.705 sec)
INFO:tensorflow:global_step/sec: 148.363
INFO:tensorflow:loss = 140163.0, step = 401 (0.675 sec)
INFO:tensorflow:global_step/sec: 131.501
INFO:tensorflow:loss = 141613.0, step = 501 (0.763 sec)
INFO:tensorflow:global_step/sec: 148.133
INFO:tensorflow:loss = 136004.0, step = 601 (0.671 sec)
INFO:tensorflow:global_step/sec: 164.039
INFO:tensorflow:loss = 130641.0, step = 701 (0.609 sec)
INFO:tensorflow:global_step/sec: 154.876
INFO:tensorflow:loss = 111425.0, step = 801 (0.647 sec)
INFO:tensorflow:global_step/sec: 161.257
INFO:tensorflow:loss = 113015.0, step = 901 (0.619 sec)
INFO:tensorflow:global_step/sec: 164.882
INFO:tensorflow:loss = 93464.3, step = 1001 (0.607 sec)
INFO:tensorflow:global_step/sec: 155.051
INFO:tensorflow:loss = 87641.5, step = 1101 (0.645 sec)
INFO:tensorflow:global_step/sec: 132.637
INFO:tensorflow:loss = 88079.6, step = 1201 (0.760 sec)
INFO:tensorflow:global_step/sec: 133.662
INFO:tensorflow:loss = 85597.5, step = 1301 (0.742 sec)
INFO:tensorflow:global_step/sec: 131.989
INFO:tensorflow:loss = 63857.3, step = 1401 (0.762 sec)
INFO:tensorflow:global_step/sec: 105.007
INFO:tensorflow:loss = 70258.2, step = 1501 (0.949 sec)
INFO:tensorflow:global_step/sec: 131.114
INFO:tensorflow:loss = 55564.2, step = 1601 (0.765 sec)
INFO:tensorflow:global_step/sec: 128.256
INFO:tensorflow:loss = 63768.5, step = 1701 (0.777 sec)
INFO:tensorflow:global_step/sec: 144.842
INFO:tensorflow:loss = 54877.1, step = 1801 (0.691 sec)
INFO:tensorflow:global_step/sec: 126.825
INFO:tensorflow:loss = 60213.6, step = 1901 (0.792 sec)
INFO:tensorflow:global_step/sec: 143.734
INFO:tensorflow:loss = 54891.5, step = 2001 (0.692 sec)
INFO:tensorflow:global_step/sec: 146.572
INFO:tensorflow:loss = 47515.1, step = 2101 (0.685 sec)
INFO:tensorflow:global_step/sec: 117.477
INFO:tensorflow:loss = 51576.9, step = 2201 (0.849 sec)
INFO:tensorflow:global_step/sec: 144.71
INFO:tensorflow:loss = 50460.4, step = 2301 (0.693 sec)
INFO:tensorflow:Saving checkpoints for 2401 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 93.9596

* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:32:07
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-2401
INFO:tensorflow:Finished evaluation at 2017-11-11-14:32:08
INFO:tensorflow:Saving dict for global step 2401: average_loss = 104.677, global_step = 2401, loss = 52338.7
INFO:tensorflow:Validation (step 2401): average_loss = 104.677, loss = 52338.7, global_step = 2401
INFO:tensorflow:loss = 54241.1, step = 2401 (1.982 sec)
INFO:tensorflow:global_step/sec: 64.2617
INFO:tensorflow:loss = 42089.1, step = 2501 (0.636 sec)
INFO:tensorflow:global_step/sec: 156.711
INFO:tensorflow:loss = 43037.2, step = 2601 (0.639 sec)
INFO:tensorflow:global_step/sec: 149.293
INFO:tensorflow:loss = 46601.3, step = 2701 (0.670 sec)
INFO:tensorflow:global_step/sec: 147.748
INFO:tensorflow:loss = 42919.7, step = 2801 (0.676 sec)
INFO:tensorflow:global_step/sec: 152.002
INFO:tensorflow:loss = 44527.5, step = 2901 (0.659 sec)
INFO:tensorflow:global_step/sec: 144.838
INFO:tensorflow:loss = 47868.0, step = 3001 (0.689 sec)
INFO:tensorflow:global_step/sec: 160.648
INFO:tensorflow:loss = 47298.0, step = 3101 (0.622 sec)
INFO:tensorflow:global_step/sec: 161.367
INFO:tensorflow:loss = 49777.9, step = 3201 (0.620 sec)
INFO:tensorflow:global_step/sec: 153.168
INFO:tensorflow:loss = 44413.0, step = 3301 (0.653 sec)
INFO:tensorflow:global_step/sec: 158.438
INFO:tensorflow:loss = 47032.9, step = 3401 (0.631 sec)
INFO:tensorflow:global_step/sec: 119.087
INFO:tensorflow:loss = 36708.7, step = 3501 (0.845 sec)
INFO:tensorflow:global_step/sec: 147.487
INFO:tensorflow:loss = 48602.3, step = 3601 (0.673 sec)
INFO:tensorflow:global_step/sec: 151.377
INFO:tensorflow:loss = 46881.4, step = 3701 (0.662 sec)
INFO:tensorflow:global_step/sec: 158.061
INFO:tensorflow:loss = 44374.4, step = 3801 (0.632 sec)
INFO:tensorflow:global_step/sec: 160.813
INFO:tensorflow:loss = 45259.9, step = 3901 (0.622 sec)
INFO:tensorflow:global_step/sec: 162.422
INFO:tensorflow:loss = 50385.1, step = 4001 (0.615 sec)
INFO:tensorflow:global_step/sec: 161.446
INFO:tensorflow:loss = 46569.6, step = 4101 (0.620 sec)
INFO:tensorflow:global_step/sec: 154.099
INFO:tensorflow:loss = 47248.7, step = 4201 (0.649 sec)
INFO:tensorflow:global_step/sec: 157.475
INFO:tensorflow:loss = 40043.4, step = 4301 (0.635 sec)
INFO:tensorflow:global_step/sec: 132.105
INFO:tensorflow:loss = 45197.2, step = 4401 (0.757 sec)
INFO:tensorflow:global_step/sec: 154.426
INFO:tensorflow:loss = 47951.4, step = 4501 (0.647 sec)
INFO:tensorflow:global_step/sec: 136.858
INFO:tensorflow:loss = 44457.1, step = 4601 (0.734 sec)
INFO:tensorflow:global_step/sec: 119.193
INFO:tensorflow:loss = 37337.1, step = 4701 (0.839 sec)
INFO:tensorflow:Saving checkpoints for 4801 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 72.1377

* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:32:25
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-4801
INFO:tensorflow:Finished evaluation at 2017-11-11-14:32:26
INFO:tensorflow:Saving dict for global step 4801: average_loss = 96.7373, global_step = 4801, loss = 48368.7
INFO:tensorflow:Validation (step 4801): average_loss = 96.7373, loss = 48368.7, global_step = 4801
INFO:tensorflow:loss = 46016.4, step = 4801 (2.349 sec)
INFO:tensorflow:global_step/sec: 60.8736
INFO:tensorflow:loss = 46275.1, step = 4901 (0.679 sec)
INFO:tensorflow:global_step/sec: 151.574
INFO:tensorflow:loss = 33856.8, step = 5001 (0.658 sec)
INFO:tensorflow:global_step/sec: 162.539
INFO:tensorflow:loss = 58067.3, step = 5101 (0.616 sec)
INFO:tensorflow:global_step/sec: 161.898
INFO:tensorflow:loss = 36591.6, step = 5201 (0.617 sec)
INFO:tensorflow:global_step/sec: 161.238
INFO:tensorflow:loss = 49899.2, step = 5301 (0.621 sec)
INFO:tensorflow:global_step/sec: 161.159
INFO:tensorflow:loss = 59058.9, step = 5401 (0.620 sec)
INFO:tensorflow:global_step/sec: 162.119
INFO:tensorflow:loss = 39515.4, step = 5501 (0.617 sec)
INFO:tensorflow:global_step/sec: 160.662
INFO:tensorflow:loss = 37766.9, step = 5601 (0.622 sec)
INFO:tensorflow:global_step/sec: 160.995
INFO:tensorflow:loss = 45288.4, step = 5701 (0.621 sec)
INFO:tensorflow:global_step/sec: 159.086
INFO:tensorflow:loss = 41809.8, step = 5801 (0.629 sec)
INFO:tensorflow:global_step/sec: 153.174
INFO:tensorflow:loss = 42418.6, step = 5901 (0.652 sec)
INFO:tensorflow:global_step/sec: 136.012
INFO:tensorflow:loss = 49231.9, step = 6001 (0.736 sec)
INFO:tensorflow:global_step/sec: 147.14
INFO:tensorflow:loss = 37564.2, step = 6101 (0.679 sec)
INFO:tensorflow:global_step/sec: 158.619
INFO:tensorflow:loss = 42841.4, step = 6201 (0.630 sec)
INFO:tensorflow:global_step/sec: 146.474
INFO:tensorflow:loss = 47508.5, step = 6301 (0.683 sec)
INFO:tensorflow:global_step/sec: 160.396
INFO:tensorflow:loss = 44208.4, step = 6401 (0.624 sec)
INFO:tensorflow:global_step/sec: 148.18
INFO:tensorflow:loss = 40910.1, step = 6501 (0.675 sec)
INFO:tensorflow:global_step/sec: 138.929
INFO:tensorflow:loss = 50287.4, step = 6601 (0.720 sec)
INFO:tensorflow:global_step/sec: 151.384
INFO:tensorflow:loss = 39495.4, step = 6701 (0.660 sec)
INFO:tensorflow:global_step/sec: 149.712
INFO:tensorflow:loss = 43742.4, step = 6801 (0.668 sec)
INFO:tensorflow:global_step/sec: 121.472
INFO:tensorflow:loss = 44626.8, step = 6901 (0.827 sec)
INFO:tensorflow:global_step/sec: 134.407
INFO:tensorflow:loss = 40497.6, step = 7001 (0.742 sec)
INFO:tensorflow:global_step/sec: 135.498
INFO:tensorflow:loss = 37368.0, step = 7101 (0.740 sec)
INFO:tensorflow:Saving checkpoints for 7201 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 76.9883

* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:32:43
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-7201
INFO:tensorflow:Finished evaluation at 2017-11-11-14:32:43
INFO:tensorflow:Saving dict for global step 7201: average_loss = 95.7069, global_step = 7201, loss = 47853.4
INFO:tensorflow:Validation (step 7201): average_loss = 95.7069, loss = 47853.4, global_step = 7201
INFO:tensorflow:loss = 51782.6, step = 7201 (2.226 sec)
INFO:tensorflow:global_step/sec: 60.5837
INFO:tensorflow:loss = 42918.2, step = 7301 (0.720 sec)
INFO:tensorflow:global_step/sec: 148.242
INFO:tensorflow:loss = 40595.4, step = 7401 (0.675 sec)
INFO:tensorflow:global_step/sec: 141.722
INFO:tensorflow:loss = 29395.8, step = 7501 (0.706 sec)
INFO:tensorflow:global_step/sec: 145.886
INFO:tensorflow:loss = 51844.5, step = 7601 (0.687 sec)
INFO:tensorflow:global_step/sec: 141.609
INFO:tensorflow:loss = 43390.5, step = 7701 (0.704 sec)
INFO:tensorflow:global_step/sec: 126.915
INFO:tensorflow:loss = 47932.0, step = 7801 (0.787 sec)
INFO:tensorflow:global_step/sec: 144.669
INFO:tensorflow:loss = 50953.5, step = 7901 (0.692 sec)
INFO:tensorflow:global_step/sec: 121.705
INFO:tensorflow:loss = 45364.8, step = 8001 (0.821 sec)
INFO:tensorflow:global_step/sec: 143.123
INFO:tensorflow:loss = 37158.0, step = 8101 (0.700 sec)
INFO:tensorflow:global_step/sec: 129.806
INFO:tensorflow:loss = 51231.4, step = 8201 (0.772 sec)
INFO:tensorflow:global_step/sec: 135.586
INFO:tensorflow:loss = 45554.2, step = 8301 (0.737 sec)
INFO:tensorflow:global_step/sec: 139.29
INFO:tensorflow:loss = 38198.0, step = 8401 (0.715 sec)
INFO:tensorflow:global_step/sec: 126.346
INFO:tensorflow:loss = 37683.4, step = 8501 (0.793 sec)
INFO:tensorflow:global_step/sec: 142.809
INFO:tensorflow:loss = 48373.8, step = 8601 (0.701 sec)
INFO:tensorflow:global_step/sec: 133.683
INFO:tensorflow:loss = 39778.3, step = 8701 (0.748 sec)
INFO:tensorflow:global_step/sec: 135.688
INFO:tensorflow:loss = 46058.1, step = 8801 (0.737 sec)
INFO:tensorflow:global_step/sec: 135.275
INFO:tensorflow:loss = 46736.0, step = 8901 (0.738 sec)
INFO:tensorflow:global_step/sec: 129.718
INFO:tensorflow:loss = 40734.2, step = 9001 (0.771 sec)
INFO:tensorflow:global_step/sec: 143.523
INFO:tensorflow:loss = 43216.0, step = 9101 (0.698 sec)
INFO:tensorflow:global_step/sec: 134.956
INFO:tensorflow:loss = 51448.4, step = 9201 (0.740 sec)
INFO:tensorflow:global_step/sec: 144.751
INFO:tensorflow:loss = 46285.5, step = 9301 (0.690 sec)
INFO:tensorflow:global_step/sec: 129.764
INFO:tensorflow:loss = 45954.6, step = 9401 (0.771 sec)
INFO:tensorflow:global_step/sec: 145.96
INFO:tensorflow:loss = 48068.8, step = 9501 (0.685 sec)
INFO:tensorflow:Saving checkpoints for 9601 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 75.1645

* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:33:02
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-9601
INFO:tensorflow:Finished evaluation at 2017-11-11-14:33:02
INFO:tensorflow:Saving dict for global step 9601: average_loss = 95.0352, global_step = 9601, loss = 47517.6
INFO:tensorflow:Validation (step 9601): average_loss = 95.0352, loss = 47517.6, global_step = 9601
INFO:tensorflow:loss = 53438.2, step = 9601 (2.298 sec)
INFO:tensorflow:global_step/sec: 59.2006
INFO:tensorflow:loss = 53259.5, step = 9701 (0.721 sec)
INFO:tensorflow:global_step/sec: 141.779
INFO:tensorflow:loss = 34251.2, step = 9801 (0.706 sec)
INFO:tensorflow:global_step/sec: 131.811
INFO:tensorflow:loss = 42877.2, step = 9901 (0.763 sec)
INFO:tensorflow:global_step/sec: 115.783
INFO:tensorflow:loss = 49685.0, step = 10001 (0.862 sec)
INFO:tensorflow:global_step/sec: 127.013
INFO:tensorflow:loss = 47262.9, step = 10101 (0.784 sec)
INFO:tensorflow:global_step/sec: 151.498
INFO:tensorflow:loss = 49317.5, step = 10201 (0.661 sec)
INFO:tensorflow:global_step/sec: 143.846
INFO:tensorflow:loss = 41488.7, step = 10301 (0.694 sec)
INFO:tensorflow:global_step/sec: 140.025
INFO:tensorflow:loss = 40965.7, step = 10401 (0.714 sec)
INFO:tensorflow:global_step/sec: 151.54
INFO:tensorflow:loss = 34246.2, step = 10501 (0.660 sec)
INFO:tensorflow:global_step/sec: 141.593
INFO:tensorflow:loss = 41920.2, step = 10601 (0.707 sec)
INFO:tensorflow:global_step/sec: 142.479
INFO:tensorflow:loss = 40829.2, step = 10701 (0.702 sec)
INFO:tensorflow:global_step/sec: 146.329
INFO:tensorflow:loss = 53730.8, step = 10801 (0.683 sec)
INFO:tensorflow:global_step/sec: 143.41
INFO:tensorflow:loss = 39976.0, step = 10901 (0.698 sec)
INFO:tensorflow:global_step/sec: 141.968
INFO:tensorflow:loss = 42867.9, step = 11001 (0.704 sec)
INFO:tensorflow:global_step/sec: 145.57
INFO:tensorflow:loss = 44854.7, step = 11101 (0.687 sec)
INFO:tensorflow:global_step/sec: 135.972
INFO:tensorflow:loss = 43357.3, step = 11201 (0.735 sec)
INFO:tensorflow:global_step/sec: 145.339
INFO:tensorflow:loss = 44547.3, step = 11301 (0.688 sec)
INFO:tensorflow:global_step/sec: 142.515
INFO:tensorflow:loss = 41676.9, step = 11401 (0.703 sec)
INFO:tensorflow:global_step/sec: 145.965
INFO:tensorflow:loss = 30990.5, step = 11501 (0.685 sec)
INFO:tensorflow:global_step/sec: 144.552
INFO:tensorflow:loss = 50680.9, step = 11601 (0.691 sec)
INFO:tensorflow:global_step/sec: 138.256
INFO:tensorflow:loss = 41594.9, step = 11701 (0.726 sec)
INFO:tensorflow:global_step/sec: 129.769
INFO:tensorflow:loss = 42287.2, step = 11801 (0.769 sec)
INFO:tensorflow:global_step/sec: 131.017
INFO:tensorflow:loss = 49489.5, step = 11901 (0.763 sec)
INFO:tensorflow:Saving checkpoints for 12001 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 74.3397

* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:33:21
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-12001
INFO:tensorflow:Finished evaluation at 2017-11-11-14:33:21
INFO:tensorflow:Saving dict for global step 12001: average_loss = 94.4759, global_step = 12001, loss = 47237.9
INFO:tensorflow:Validation (step 12001): average_loss = 94.4759, loss = 47237.9, global_step = 12001
INFO:tensorflow:loss = 48240.1, step = 12001 (2.347 sec)
INFO:tensorflow:global_step/sec: 54.1707
INFO:tensorflow:loss = 33612.9, step = 12101 (0.847 sec)
INFO:tensorflow:global_step/sec: 124.694
INFO:tensorflow:loss = 38861.4, step = 12201 (0.802 sec)
INFO:tensorflow:global_step/sec: 118.321
INFO:tensorflow:loss = 42655.6, step = 12301 (0.845 sec)
INFO:tensorflow:global_step/sec: 124.359
INFO:tensorflow:loss = 35384.6, step = 12401 (0.803 sec)
INFO:tensorflow:global_step/sec: 127.487
INFO:tensorflow:loss = 43430.5, step = 12501 (0.785 sec)
INFO:tensorflow:global_step/sec: 140.518
INFO:tensorflow:loss = 48275.4, step = 12601 (0.710 sec)
INFO:tensorflow:global_step/sec: 142.918
INFO:tensorflow:loss = 37730.8, step = 12701 (0.700 sec)
INFO:tensorflow:global_step/sec: 143.119
INFO:tensorflow:loss = 48645.6, step = 12801 (0.701 sec)
INFO:tensorflow:global_step/sec: 138.138
INFO:tensorflow:loss = 30703.5, step = 12901 (0.722 sec)
INFO:tensorflow:global_step/sec: 145.613
INFO:tensorflow:loss = 43611.7, step = 13001 (0.687 sec)
INFO:tensorflow:global_step/sec: 144.863
INFO:tensorflow:loss = 38099.9, step = 13101 (0.691 sec)
INFO:tensorflow:global_step/sec: 137.656
INFO:tensorflow:loss = 45194.5, step = 13201 (0.728 sec)
INFO:tensorflow:global_step/sec: 122.444
INFO:tensorflow:loss = 38871.8, step = 13301 (0.815 sec)
INFO:tensorflow:global_step/sec: 138.867
INFO:tensorflow:loss = 36956.7, step = 13401 (0.720 sec)
INFO:tensorflow:global_step/sec: 124.395
INFO:tensorflow:loss = 39388.8, step = 13501 (0.803 sec)
INFO:tensorflow:global_step/sec: 157.459
INFO:tensorflow:loss = 46446.1, step = 13601 (0.635 sec)
INFO:tensorflow:global_step/sec: 148.935
INFO:tensorflow:loss = 44792.6, step = 13701 (0.672 sec)
INFO:tensorflow:global_step/sec: 138.586
INFO:tensorflow:loss = 55869.1, step = 13801 (0.721 sec)
INFO:tensorflow:global_step/sec: 152.045
INFO:tensorflow:loss = 42912.5, step = 13901 (0.658 sec)
INFO:tensorflow:global_step/sec: 127.431
INFO:tensorflow:loss = 44276.9, step = 14001 (0.786 sec)
INFO:tensorflow:global_step/sec: 128.268
INFO:tensorflow:loss = 32231.6, step = 14101 (0.780 sec)
INFO:tensorflow:global_step/sec: 136.64
INFO:tensorflow:loss = 33367.5, step = 14201 (0.730 sec)
INFO:tensorflow:global_step/sec: 146.577
INFO:tensorflow:loss = 43804.7, step = 14301 (0.682 sec)
INFO:tensorflow:Saving checkpoints for 14401 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 77.9368

* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:33:40
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-14401
INFO:tensorflow:Finished evaluation at 2017-11-11-14:33:40
INFO:tensorflow:Saving dict for global step 14401: average_loss = 93.9907, global_step = 14401, loss = 46995.4
INFO:tensorflow:Validation (step 14401): average_loss = 93.9907, loss = 46995.4, global_step = 14401
INFO:tensorflow:loss = 57803.0, step = 14401 (2.139 sec)
INFO:tensorflow:global_step/sec: 61.7375
INFO:tensorflow:loss = 46377.5, step = 14501 (0.764 sec)
INFO:tensorflow:global_step/sec: 124.959
INFO:tensorflow:loss = 34797.7, step = 14601 (0.801 sec)
INFO:tensorflow:global_step/sec: 144.906
INFO:tensorflow:loss = 44715.1, step = 14701 (0.690 sec)
INFO:tensorflow:global_step/sec: 143.668
INFO:tensorflow:loss = 39205.6, step = 14801 (0.695 sec)
INFO:tensorflow:global_step/sec: 147.122
INFO:tensorflow:loss = 40149.7, step = 14901 (0.679 sec)
INFO:tensorflow:global_step/sec: 147.896
INFO:tensorflow:loss = 43184.4, step = 15001 (0.677 sec)
INFO:tensorflow:global_step/sec: 146.5
INFO:tensorflow:loss = 38481.5, step = 15101 (0.682 sec)
INFO:tensorflow:global_step/sec: 144.174
INFO:tensorflow:loss = 40600.9, step = 15201 (0.694 sec)
INFO:tensorflow:global_step/sec: 141.158
INFO:tensorflow:loss = 49492.6, step = 15301 (0.709 sec)
INFO:tensorflow:global_step/sec: 132.475
INFO:tensorflow:loss = 29930.4, step = 15401 (0.754 sec)
INFO:tensorflow:global_step/sec: 147.538
INFO:tensorflow:loss = 46129.1, step = 15501 (0.678 sec)
INFO:tensorflow:global_step/sec: 143.941
INFO:tensorflow:loss = 47175.0, step = 15601 (0.696 sec)
INFO:tensorflow:global_step/sec: 144.395
INFO:tensorflow:loss = 39116.7, step = 15701 (0.692 sec)
INFO:tensorflow:global_step/sec: 140.103
INFO:tensorflow:loss = 35493.4, step = 15801 (0.713 sec)
INFO:tensorflow:global_step/sec: 148.264
INFO:tensorflow:loss = 41123.8, step = 15901 (0.674 sec)
INFO:tensorflow:global_step/sec: 141.081
INFO:tensorflow:loss = 28964.1, step = 16001 (0.709 sec)
INFO:tensorflow:global_step/sec: 129.559
INFO:tensorflow:loss = 42644.8, step = 16101 (0.773 sec)
INFO:tensorflow:global_step/sec: 117.856
INFO:tensorflow:loss = 41032.1, step = 16201 (0.849 sec)
INFO:tensorflow:global_step/sec: 126.27
INFO:tensorflow:loss = 38422.3, step = 16301 (0.793 sec)
INFO:tensorflow:global_step/sec: 133.53
INFO:tensorflow:loss = 40377.9, step = 16401 (0.746 sec)
INFO:tensorflow:global_step/sec: 146.292
INFO:tensorflow:loss = 45031.1, step = 16501 (0.685 sec)
INFO:tensorflow:global_step/sec: 128.616
INFO:tensorflow:loss = 44225.1, step = 16601 (0.776 sec)
INFO:tensorflow:global_step/sec: 119.769
INFO:tensorflow:loss = 44445.1, step = 16701 (0.836 sec)
INFO:tensorflow:Saving checkpoints for 16801 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 76.8777

* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:33:59
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-16801
INFO:tensorflow:Finished evaluation at 2017-11-11-14:33:59
INFO:tensorflow:Saving dict for global step 16801: average_loss = 93.6571, global_step = 16801, loss = 46828.6
INFO:tensorflow:Validation (step 16801): average_loss = 93.6571, loss = 46828.6, global_step = 16801
INFO:tensorflow:loss = 43306.4, step = 16801 (2.293 sec)
INFO:tensorflow:global_step/sec: 55.6313
INFO:tensorflow:loss = 54241.1, step = 16901 (0.808 sec)
INFO:tensorflow:global_step/sec: 139.604
INFO:tensorflow:loss = 34022.9, step = 17001 (0.715 sec)
INFO:tensorflow:global_step/sec: 139.468
INFO:tensorflow:loss = 45510.8, step = 17101 (0.716 sec)
INFO:tensorflow:global_step/sec: 142.418
INFO:tensorflow:loss = 43210.0, step = 17201 (0.702 sec)
INFO:tensorflow:global_step/sec: 143.537
INFO:tensorflow:loss = 50877.1, step = 17301 (0.697 sec)
INFO:tensorflow:global_step/sec: 135.489
INFO:tensorflow:loss = 38702.2, step = 17401 (0.739 sec)
INFO:tensorflow:global_step/sec: 145.958
INFO:tensorflow:loss = 34215.1, step = 17501 (0.684 sec)
INFO:tensorflow:global_step/sec: 153.087
INFO:tensorflow:loss = 39389.1, step = 17601 (0.654 sec)
INFO:tensorflow:global_step/sec: 133.414
INFO:tensorflow:loss = 43825.1, step = 17701 (0.750 sec)
INFO:tensorflow:global_step/sec: 132.724
INFO:tensorflow:loss = 43972.8, step = 17801 (0.754 sec)
INFO:tensorflow:global_step/sec: 138.88
INFO:tensorflow:loss = 48032.7, step = 17901 (0.721 sec)
INFO:tensorflow:global_step/sec: 131.104
INFO:tensorflow:loss = 43332.0, step = 18001 (0.761 sec)
INFO:tensorflow:global_step/sec: 129.382
INFO:tensorflow:loss = 39407.1, step = 18101 (0.772 sec)
INFO:tensorflow:global_step/sec: 146.652
INFO:tensorflow:loss = 46066.7, step = 18201 (0.681 sec)
INFO:tensorflow:global_step/sec: 157.141
INFO:tensorflow:loss = 43000.3, step = 18301 (0.637 sec)
INFO:tensorflow:global_step/sec: 134.653
INFO:tensorflow:loss = 43819.3, step = 18401 (0.743 sec)
INFO:tensorflow:global_step/sec: 146.891
INFO:tensorflow:loss = 43357.4, step = 18501 (0.680 sec)
INFO:tensorflow:global_step/sec: 155.775
INFO:tensorflow:loss = 50256.3, step = 18601 (0.642 sec)
INFO:tensorflow:global_step/sec: 135.905
INFO:tensorflow:loss = 42203.0, step = 18701 (0.736 sec)
INFO:tensorflow:global_step/sec: 150.761
INFO:tensorflow:loss = 39352.0, step = 18801 (0.663 sec)
INFO:tensorflow:global_step/sec: 156.62
INFO:tensorflow:loss = 44110.5, step = 18901 (0.638 sec)
INFO:tensorflow:global_step/sec: 148.473
INFO:tensorflow:loss = 52128.4, step = 19001 (0.674 sec)
INFO:tensorflow:global_step/sec: 131.937
INFO:tensorflow:loss = 44822.1, step = 19101 (0.758 sec)
INFO:tensorflow:Saving checkpoints for 19201 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 75.6735

* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:34:18
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-19201
INFO:tensorflow:Finished evaluation at 2017-11-11-14:34:18
INFO:tensorflow:Saving dict for global step 19201: average_loss = 93.4035, global_step = 19201, loss = 46701.7
INFO:tensorflow:Validation (step 19201): average_loss = 93.4035, loss = 46701.7, global_step = 19201
INFO:tensorflow:loss = 41447.1, step = 19201 (2.250 sec)
INFO:tensorflow:global_step/sec: 60.5112
INFO:tensorflow:loss = 39285.7, step = 19301 (0.724 sec)
INFO:tensorflow:global_step/sec: 142.25
INFO:tensorflow:loss = 47136.1, step = 19401 (0.704 sec)
INFO:tensorflow:global_step/sec: 143.985
INFO:tensorflow:loss = 48500.1, step = 19501 (0.693 sec)
INFO:tensorflow:global_step/sec: 145.566
INFO:tensorflow:loss = 41969.5, step = 19601 (0.689 sec)
INFO:tensorflow:global_step/sec: 126.264
INFO:tensorflow:loss = 45881.1, step = 19701 (0.789 sec)
INFO:tensorflow:global_step/sec: 149.807
INFO:tensorflow:loss = 38483.6, step = 19801 (0.668 sec)
INFO:tensorflow:global_step/sec: 147.313
INFO:tensorflow:loss = 38991.4, step = 19901 (0.679 sec)
INFO:tensorflow:global_step/sec: 154.983
INFO:tensorflow:loss = 33769.5, step = 20001 (0.645 sec)
INFO:tensorflow:global_step/sec: 135.423
INFO:tensorflow:loss = 37037.8, step = 20101 (0.738 sec)
INFO:tensorflow:global_step/sec: 143.824
INFO:tensorflow:loss = 44271.2, step = 20201 (0.698 sec)
INFO:tensorflow:global_step/sec: 142.841
INFO:tensorflow:loss = 47297.9, step = 20301 (0.699 sec)
INFO:tensorflow:global_step/sec: 136.822
INFO:tensorflow:loss = 49067.2, step = 20401 (0.730 sec)
INFO:tensorflow:global_step/sec: 148.37
INFO:tensorflow:loss = 29920.0, step = 20501 (0.673 sec)
INFO:tensorflow:global_step/sec: 155.2
INFO:tensorflow:loss = 41268.2, step = 20601 (0.645 sec)
INFO:tensorflow:global_step/sec: 150.811
INFO:tensorflow:loss = 48263.5, step = 20701 (0.664 sec)
INFO:tensorflow:global_step/sec: 152.439
INFO:tensorflow:loss = 43600.9, step = 20801 (0.655 sec)
INFO:tensorflow:global_step/sec: 154.507
INFO:tensorflow:loss = 53666.0, step = 20901 (0.647 sec)
INFO:tensorflow:global_step/sec: 140.796
INFO:tensorflow:loss = 43610.4, step = 21001 (0.711 sec)
INFO:tensorflow:global_step/sec: 147.657
INFO:tensorflow:loss = 39353.9, step = 21101 (0.677 sec)
INFO:tensorflow:global_step/sec: 150.085
INFO:tensorflow:loss = 41081.1, step = 21201 (0.667 sec)
INFO:tensorflow:global_step/sec: 127.016
INFO:tensorflow:loss = 43726.5, step = 21301 (0.787 sec)
INFO:tensorflow:global_step/sec: 135.764
INFO:tensorflow:loss = 46147.2, step = 21401 (0.737 sec)
INFO:tensorflow:global_step/sec: 136.951
INFO:tensorflow:loss = 42668.0, step = 21501 (0.731 sec)
INFO:tensorflow:Saving checkpoints for 21601 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 73.8537

* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:34:36
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-21601
INFO:tensorflow:Finished evaluation at 2017-11-11-14:34:36
INFO:tensorflow:Saving dict for global step 21601: average_loss = 93.2296, global_step = 21601, loss = 46614.8
INFO:tensorflow:Validation (step 21601): average_loss = 93.2296, loss = 46614.8, global_step = 21601
INFO:tensorflow:loss = 39012.0, step = 21601 (2.317 sec)
INFO:tensorflow:global_step/sec: 57.3267
INFO:tensorflow:loss = 42405.3, step = 21701 (0.783 sec)
INFO:tensorflow:global_step/sec: 141.019
INFO:tensorflow:loss = 47157.8, step = 21801 (0.707 sec)
INFO:tensorflow:global_step/sec: 143.884
INFO:tensorflow:loss = 40285.0, step = 21901 (0.695 sec)
INFO:tensorflow:global_step/sec: 130.032
INFO:tensorflow:loss = 51357.3, step = 22001 (0.769 sec)
INFO:tensorflow:global_step/sec: 150.628
INFO:tensorflow:loss = 43420.6, step = 22101 (0.663 sec)
INFO:tensorflow:global_step/sec: 134.978
INFO:tensorflow:loss = 50706.4, step = 22201 (0.741 sec)
INFO:tensorflow:global_step/sec: 138.679
INFO:tensorflow:loss = 47902.4, step = 22301 (0.721 sec)
INFO:tensorflow:global_step/sec: 137.255
INFO:tensorflow:loss = 34552.7, step = 22401 (0.730 sec)
INFO:tensorflow:global_step/sec: 139.58
INFO:tensorflow:loss = 40107.3, step = 22501 (0.715 sec)
INFO:tensorflow:global_step/sec: 122.743
INFO:tensorflow:loss = 43026.8, step = 22601 (0.817 sec)
INFO:tensorflow:global_step/sec: 118.511
INFO:tensorflow:loss = 38352.9, step = 22701 (0.846 sec)
INFO:tensorflow:global_step/sec: 127.974
INFO:tensorflow:loss = 47005.3, step = 22801 (0.781 sec)
INFO:tensorflow:global_step/sec: 132.907
INFO:tensorflow:loss = 34692.8, step = 22901 (0.751 sec)
INFO:tensorflow:global_step/sec: 145.023
INFO:tensorflow:loss = 45891.2, step = 23001 (0.689 sec)
INFO:tensorflow:global_step/sec: 143.874
INFO:tensorflow:loss = 44916.1, step = 23101 (0.695 sec)
INFO:tensorflow:global_step/sec: 132.35
INFO:tensorflow:loss = 42158.5, step = 23201 (0.755 sec)
INFO:tensorflow:global_step/sec: 119.94
INFO:tensorflow:loss = 34986.8, step = 23301 (0.838 sec)
INFO:tensorflow:global_step/sec: 124.828
INFO:tensorflow:loss = 49937.9, step = 23401 (0.798 sec)
INFO:tensorflow:global_step/sec: 129.237
INFO:tensorflow:loss = 51601.8, step = 23501 (0.773 sec)
INFO:tensorflow:global_step/sec: 135.224
INFO:tensorflow:loss = 44515.1, step = 23601 (0.740 sec)
INFO:tensorflow:global_step/sec: 140.371
INFO:tensorflow:loss = 43017.3, step = 23701 (0.713 sec)
INFO:tensorflow:global_step/sec: 122.712
INFO:tensorflow:loss = 34417.2, step = 23801 (0.817 sec)
INFO:tensorflow:global_step/sec: 133.027
INFO:tensorflow:loss = 45028.1, step = 23901 (0.750 sec)
INFO:tensorflow:Saving checkpoints for 24000 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:Loss for final step: 41759.8.

* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:34:56
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-24000
INFO:tensorflow:Finished evaluation at 2017-11-11-14:34:56
INFO:tensorflow:Saving dict for global step 24000: average_loss = 93.1534, global_step = 24000, loss = 46576.7
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-24000
INFO:tensorflow:Assets added to graph.
INFO:tensorflow:No assets to write.
INFO:tensorflow:SavedModel written to: b"trained_models/reg-model-04/export/Servo/temp-b'1510410896'/saved_model.pb"
.......................................
Experiment finished at 14:34:57

Experiment elapsed time: 193.639007 seconds

7. Evaluate the Model





In [13]:



    


TRAIN_SIZE = 12000
VALID_SIZE = 3000
TEST_SIZE = 5000

train_input_fn = lambda: csv_input_fn(files_name_pattern= TRAIN_DATA_FILES_PATTERN, 
                                      mode= tf.estimator.ModeKeys.EVAL,
                                      batch_size= TRAIN_SIZE)

valid_input_fn = lambda: csv_input_fn(files_name_pattern= VALID_DATA_FILES_PATTERN, 
                                      mode= tf.estimator.ModeKeys.EVAL,
                                      batch_size= VALID_SIZE)

test_input_fn = lambda: csv_input_fn(files_name_pattern= TEST_DATA_FILES_PATTERN, 
                                      mode= tf.estimator.ModeKeys.EVAL,
                                      batch_size= TEST_SIZE)

estimator = create_estimator(run_config, hparams)

train_results = estimator.evaluate(input_fn=train_input_fn, steps=1)
train_rmse = round(math.sqrt(train_results["average_loss"]),5)
print()
print("############################################################################################")
print("# Train RMSE: {} - {}".format(train_rmse, train_results))
print("############################################################################################")

valid_results = estimator.evaluate(input_fn=valid_input_fn, steps=1)
valid_rmse = round(math.sqrt(valid_results["average_loss"]),5)
print()
print("############################################################################################")
print("# Valid RMSE: {} - {}".format(valid_rmse,valid_results))
print("############################################################################################")

test_results = estimator.evaluate(input_fn=test_input_fn, steps=1)
test_rmse = round(math.sqrt(test_results["average_loss"]),5)
print()
print("############################################################################################")
print("# Test RMSE: {} - {}".format(test_rmse, test_results))
print("############################################################################################")



    


















    






INFO:tensorflow:Using config: {'_task_type': None, '_task_id': 0, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x11e860a58>, '_master': '', '_num_ps_replicas': 0, '_num_worker_replicas': 0, '_environment': 'local', '_is_chief': True, '_evaluation_master': '', '_tf_config': gpu_options {
  per_process_gpu_memory_fraction: 1
}
, '_tf_random_seed': 19830610, '_save_summary_steps': 100, '_save_checkpoints_secs': None, '_log_step_count_steps': 100, '_session_config': None, '_save_checkpoints_steps': 2400, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_model_dir': 'trained_models/reg-model-04'}

Estimator Type: <class 'tensorflow.python.estimator.canned.dnn.DNNRegressor'>


* data input_fn:
================
Input file(s): data/train-*.csv
Batch size: 12000
Epoch Count: None
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:34:58
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-24000
INFO:tensorflow:Evaluation [1/1]
INFO:tensorflow:Finished evaluation at 2017-11-11-14:34:58
INFO:tensorflow:Saving dict for global step 24000: average_loss = 84.9214, global_step = 24000, loss = 1.01906e+06

############################################################################################
# Train RMSE: 9.21528 - {'average_loss': 84.921417, 'loss': 1019057.0, 'global_step': 24000}
############################################################################################

* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 3000
Epoch Count: None
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:34:59
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-24000
INFO:tensorflow:Evaluation [1/1]
INFO:tensorflow:Finished evaluation at 2017-11-11-14:34:59
INFO:tensorflow:Saving dict for global step 24000: average_loss = 93.1534, global_step = 24000, loss = 279460.0

############################################################################################
# Valid RMSE: 9.6516 - {'average_loss': 93.153397, 'loss': 279460.19, 'global_step': 24000}
############################################################################################

* data input_fn:
================
Input file(s): data/test-*.csv
Batch size: 5000
Epoch Count: None
Mode: eval
Shuffle: False
================

INFO:tensorflow:Starting evaluation at 2017-11-11-14:34:59
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-24000
INFO:tensorflow:Evaluation [1/1]
INFO:tensorflow:Finished evaluation at 2017-11-11-14:35:00
INFO:tensorflow:Saving dict for global step 24000: average_loss = 93.2213, global_step = 24000, loss = 466107.0

############################################################################################
# Test RMSE: 9.65512 - {'average_loss': 93.221344, 'loss': 466106.72, 'global_step': 24000}
############################################################################################

8. Prediction





In [14]:



    


import itertools

predict_input_fn = lambda: csv_input_fn(files_name_pattern=TEST_DATA_FILES_PATTERN, 
                                      mode= tf.estimator.ModeKeys.PREDICT,
                                      batch_size= 5)

predictions = estimator.predict(input_fn=predict_input_fn)
values = list(map(lambda item: item["predictions"][0],list(itertools.islice(predictions, 5))))
print()
print("Predicted Values: {}".format(values))



    


















    






* data input_fn:
================
Input file(s): data/test-*.csv
Batch size: 5
Epoch Count: None
Mode: infer
Shuffle: False
================

WARNING:tensorflow:Input graph does not contain a QueueRunner. That means predict yields forever. This is probably a mistake.
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-24000

Predicted Values: [51.019321, -5.8079214, 19.57333, 2.9324729, 1.5238302]

What can we improve?

  • Use .tfrecords files instead of CSV - TFRecord files are optimised for tensorflow.
  • Build a Custom Estimator - Custom Estimator APIs give you the flexibility to build custom models in a simple and standard way




In [ ]:

Content source: GoogleCloudPlatform/tf-estimator-tutorials

Similar notebooks:

notebook.community | gallery | about